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Abstract 


An airborne tool has been developed based on the concept that an 
aircraft maintain a time-based spacing interval from the preceding 
aircraft. The Advanced Terminal Area Approach Spacing (ATAAS) tool 
uses Automatic Dependent Surveillance-Broadcast (ADS-B) aircraft 
state data to compute a speed command for the ATAAS- equipped aircraft 
to obtain a required time interval behind another aircraft. The tool and 
candidate operational procedures were tested in a high-fidelity, full 
mission simulator with active airline subject pilots flying an arrival 
scenario by using three different modes for speed control. The primary 
objective of this study was to evaluate the concept from the standpoint of 
pilot acceptability and workload. A secondary objective was to validate 
the results of a prior Monte Carlo analysis of the ATAAS algorithm. 

Results showed that the aircraft was able to consistently achieve the 
target spacing interval within 1 s (the equivalent of approximately 220 ft 
at a final approach speed of 130 knots) when the ATAAS speed guidance 
was autothrottle-coupled and a slightly greater (4-5 s), but consistent 
interval with the pilot-controlled speed modes. The subject pilots 
generally rated the workload level with the ATAAS procedure as similar 
to that with standard procedures for a nominal Instrument Landing 
System (ITS) approach, and they also rated most aspects of the 
procedure highly in terms of acceptability. Although pilots indicated that 
the head-down time was slightly higher with ATAAS, eyetr acker data 
showed only slight changes in instrument scan patterns and no 
significant change in the amount of time spent looking out the window 
with ATAAS, versus standard ITS procedures. Pilots also rated highly the 
acceptability of the amount of head-down time required to use the 
ATAAS tool. 

1.0. Introduction 

In recent years, air travel has increased at unprecedented rates, leading to traffic congestion in many of 
the nation’s busiest terminal areas. With this trend expected to continue into the foreseeable future, many 
government and industry efforts have focused on research programs aimed at alleviating congestion 
through development of new procedures for airborne and ground-based use with supporting new 
technologies. To address this problem, the National Aeronautics and Space Administration’s (NASA) 
Advanced Air Transportation Technologies (AATT) Project developed the concept of Distributed 
Air/Ground Air Traffic Management (DAG-TM). The DAG-TM concept involves various levels of 
collaboration between airborne and ground-based resources to enable less-restricted and more efficient 
aircraft trajectories throughout all phases of flight, leading to increased airport capacity. 

The element of the DAG-TM concept that focuses on terminal area operations requires the develop- 
ment of procedures and technologies that allow aircraft to have more flexibility in choosing an efficient 
route through the terminal area, while arriving at the runway threshold properly and efficiently spaced 
from the preceding aircraft. The concept of Approach Spacing allows for a safe reduction in the excess 
spacing in traffic streams from what current procedures allow by increasing the precision with which 
aircraft are spaced, such that they can be delivered at the proper spacing interval at the runway threshold. 



This precision requires the capability to accurately predict and control the spacing intervals between 
arriving aircraft. To meet this objective, an airborne tool, the Advanced Terminal Area Approach Spacing 
(ATAAS) tool, was recently developed at NASA’s Langley Research Center (LaRC) (ref. 1). The 
ATAAS tool, a refinement of previous techniques, is based on previous work, which embodied the con- 
cept that an aircraft maintain a time-based, rather than distance-based, spacing interval from the preceding 
aircraft (refs. 2-5). 

The ATAAS tool uses Automatic Dependent Surveillance-Broadcast (ADS-B) aircraft state data, 
along with final approach speeds and wind data, to compute speed commands that the ATAAS-equipped 
aircraft must follow to achieve the required runway-threshold time interval behind the other aircraft. This 
tool has undergone extensive Monte Carlo analysis to characterize and refine its performance. Although 
the tool has many potential applications in different types of operational scenarios, including merging 
routes, enroute, and oceanic operations, the concept of in-trail spacing in the terminal area (i.e., aircraft 
are spaced longitudinally while following directly behind each other on the same ground track) was the 
logical first step in the evolution of the end-state goal of more efficient and flexible maneuvering through 
the terminal area. Future plans for research in this area will address the merge problem, in which an 
aircraft arriving in the terminal area is sequenced behind an aircraft arriving from a different direction. 

The ATAAS tool was tested with airline pilots in a high-fidelity, full mission engineering simulator to 
evaluate workload and pilot acceptability issues associated with its use and to explore the feasibility of the 
operational concept (i.e., can the assigned spacing interval be consistently achieved with the algorithm). 
This document describes the ATAAS tool, simulation test procedures, and presents the results of the testA 

Documentation for these program concepts is available at these web sites: 

NASA document: http://www.as.nasa.gov/aatt/dagconop.pdf 

NASA contractor document: http://www.asc.nasa.gov/aatt/rto/RTOFinal41_4.pdf 

2.0. Background 

2.1. Past Work 

Previous research has investigated the feasibility of using traffic information displayed on the flight 
deck to enable airborne-managed spacing (refs. 2-5). Simulator experiments conducted at LaRC that 
involve the use of Cockpit Display of Traffic Information (CDTI), including a display of the lead traffic’s 
location and other predictors on the subject aircraft’s Navigation Display (ND), found that time-based 
spacing was the most useful technique. A “time box” was used to represent the position where the subject 
aircraft (“ownship”) should be. (The position of the time box on the ND provided a target for the ownship 
to achieve.) To be at the correct spacing interval (the spacing interval was assigned by Air Traffic Control 
(ATC)) behind the aircraft it was following, the ownship had to be positioned inside the time box. The 
studies concluded that this concept was feasible from a crew workload and acceptability standpoint, 
although accurate knowledge of the positions and speeds of the aircraft with fast data update rates are 
necessary. Recent improvements in display and computing capabilities and broadcast of traffic state data 
have made the concept realizable. 

2.2. Approach Spacing Concept 

The ultimate goal behind the in-trail approach spacing concept is not to optimize precision spacing for 
individual pairs of aircraft but rather to achieve a system-wide improvement in performance. That 
improvement will be realized by obtaining better consistency in spacing from a system-wide standpoint, 

2 



sometimes at the expense of having excessive spacing between individual aircraft pairs. As such, no 
single aircraft will receive guidance to aggressively achieve a spacing interval beyond what would nor- 
mally be expected in current-day operations. Increasing the speed of one aircraft excessively to “close up 
the gap” with a preceding aircraft would quickly destabilize the system and would not increase 
system-wide performance. This destabilization could multiply the effect on the speed required of every 
aircraft that is in-trail, creating increasingly larger gaps and speeds well beyond acceptable levels by 
today’s standards. In future applications, any reduction in system throughput that could result from this 
type of limitation could be recovered through other methods, such as adjusting the lateral route in a 
designated maneuvering area. Flight crew procedures were developed to implement this in-trail concept 
with a focus on minimal effect to current workload levels. 

Part of the Approach Spacing concept vision is the ability for unequipped aircraft (i.e., those without 
an ATAAS implementation) also to participate in this operation by means of a charted arrival. Including 
the nominal routing and speed profile as part of the charted arrival allows an aircraft that can maintain the 
charted profile to be cleared for and fly this arrival. By broadcasting its position and the appropriate data, 
it can also serve as a lead aircraft for the ATAAS-equipped aircraft sequenced behind it. This concept can 
also be extended to lower-density facilities as their traffic levels increase. The procedure allows aircraft to 
perform approach spacing operations at those facilities, enabling more consistent and reliable spacing of 
arrivals with minimal changes to infrastructure. 

A fundamental issue that is unchanged from current-day procedures is the responsibility for maintain- 
ing separation between aircraft. With this new concept, responsibility remains with the Air Traffic Service 
Provider (ATSP). With this issue in mind, the clearance to conduct the approach-spacing operation is 
then a clearance to follow the ATAAS speeds. The clearance phraseology used for this study reflects this 
approach (see appendix A). 

To develop the concept of in- trail, airborne-managed spacing in the terminal area, a nominal scenario 
was defined to include system and operational (crew and controller) procedures, with candidate phrase- 
ologies and a crew interface with the ATAAS tool. The concept includes the use of a charted Standard 
Terminal Arrival Route (STAR), similar to those currently in use today. The arrival route is extended to 
include a complete lateral path to the runway, plus a vertical profile (speed and altitude), all of which 
become part of the nominal arrival clearance. The nominal speed profile associated with this charted 
procedure provides a basis around which the ATAAS algorithm will build the speed commands the flight 
crew will use. The basic system procedure is the issuance of an additional clearance from the controller to 
the ATAAS-equipped aircraft flight crew, which identifies the traffic to follow (TTF) and the assigned 
time interval for spacing. This clearance could be issued at any time during the arrival. Once the flight 
crew accepts the spacing clearance and begins following the ATAAS-commanded speeds, no further 
speed clearances are needed from the Air Traffic Service Provider (ATSP), but other normal communica- 
tions (i.e., frequency changes and approach and landing clearances) take place as usual. 

Appropriate flight crew procedures were developed to allow interaction with the ATAAS tool, with 
minimal effect to current workload levels. Supporting display elements provide information to the crew 
on the ATAAS mode and the current state of the ATAAS-equipped aircraft (“ownship”) relative to the 
aircraft (the “lead” aircraft) it is spacing behind. A simple pilot interface with the ATAAS tool allows the 
crew to select the lead aircraft and enter other appropriate data (details of the interface, displays, and crew 
procedures used in this study are provided in the Method of Test section). In future applications, data- 
linked information could replace some of these required pilot interactions. 

To test the ATAAS tool in a full-mission simulator, a nominal, in-trail arrival scenario was developed. 
Airline subject pilots were recruited to fly the simulator with ATAAS by using three different methods for 
controlling speed. Aircraft and ATAAS state and mode data were collected, pilot eye movements were 
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recorded, and pilots provided subjective ratings of perceived workload levels and various other aspects of 
the concept through questionnaires. 


2.3. ATAAS Algorithm 

The ATAAS algorithm is designed to provide pilots with speed guidance which, when properly fol- 
lowed, will result in the target spacing interval behind the lead aircraft at the runway threshold. Support- 
ing pilot interface and display elements provide information on the mode of operation and the state of the 
ATAAS-equipped aircraft (“ownship”) relative to the aircraft it is spacing behind (the “lead” aircraft). To 
achieve the concept goals for system-wide efficiency, the ATAAS algorithm was developed with features 
and limits on the speed guidance it provides. Commanded speed will not exceed 10 percent of the nomi- 
nal (charted) speed for any given segment on the arrival. Speed commands are also limited to prevent 
exceeding flap and landing gear limits. A more detailed description of the development of the ATAAS 
algorithm can be found in reference 1 . 

A trail of “history dots” behind the lead aircraft show its ground track on the ownship ’s ND and can be 
used (flown manually) instead of an “area navigation” (RNAV) route for lateral navigation. A simple 
pilot interface with the ATAAS tool allows the crew to select the lead aircraft and to enter other appropri- 
ate data required for optimizing the ATAAS tool’s performance. The aircraft speed can be controlled to 
follow the ATAAS command speed automatically by engaging the tool directly connected to the 
autothrust system or with the pilot following the displayed command speed by making appropriate inputs 
to either the throttle levers or by dialing in the command speed in the Mode Control Panel (MCP) speed 
window. These three methods of speed control are referred to in this study as Automatic, Manual, and 
MCP, respectively. The speed guidance provided by the ATAAS system does not differ in any way for 
any of the methods of speed control used. 

A feature of the ATAAS algorithm is its ability to provide a smooth transition from the commanded 
speed required for achieving the spacing interval to the final approach speed entered on the ATAAS 
Approach Data Control Display Unit (CDU) page by the flight crew. The algorithm is automatically 
switched to this approach mode near the final approach fix to allow enough time to achieve a stabilized 
final approach. When the algorithm makes a transition to this mode, it is no longer actively trying to 
maintain a spacing interval but rather is providing a scheduled speed reduction to the final approach speed 
selected by the flight crew. 

A Monte Carlo analysis that uses the traffic distribution from a typical, high volume airport was con- 
ducted by using the ATAAS algorithm. The distribution of traffic only affected the final approach speed 
variable. To exclude multimodal effects of the minimum separation distances relative to aircraft catego- 
ries, the minimum distance was set to 2.5 nmi. 

The significant parameters used in the Monte Carlo analysis were the following: 

• 50 aircraft in each arrival stream 

• 100 arrival streams 

• no winds 

• a 2-s error (1-sigma) in the pilots’ response to a change in the speed command 

• a 2-knot error (1 -sigma) in the pilots’ speed command tracking performance 

The nominal threshold crossing time was arbitrarily set to 93 s. 
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Based on the previous information, the significant results were as follows: 

(1) The average threshold crossing time was 94.04 s (an average of 1.04 s late). 

(2) The threshold crossing time error (1-sigma) was 2.49 s. 

(3) The minimum distance between aircraft pairs (1 aircraft pair in 5000) was 2.88 nm. 

Although no winds were used in this analysis, additional variations in airspeed due to winds can have 
a significant effect on the arrival time if they are not adequately accounted for in the algorithm’s compu- 
tations. The inclusion of airport wind data as a pilot-entered piece of information can mitigate this effect, 
but a more effective solution would be to include wind data as part of the ADS-B data set. 

3.0. Test Method 

3.1. Facilities 

3.1.1. Flight Simulator 

The facility used for this experiment was the NASA LaRC Integration Flight Deck (IFD) simulator 
(fig. 1). The IFD simulator cab is an engineering cab designed to represent the conventional flight deck of 
the NASA ARIES (Airborne Research Integrated Experiments System) B-757 airplane. The cab is popu- 
lated with flight instrumentation, including the overhead subsystems panels, to replicate the B-757. The 
cockpit contains a “Panorama” visual out-the-window display system. This system provides a 200° by 40° 
visual out-the-window display to add realism to piloted experiments. 



Figure 1. NASA LaRC IFD simulator. 
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During these simulation tests, significant cockpit modifications included a nonstandard control panel 
for the ND, the addition of a page to the CDU, and minor format modifications to the Electronic Attitude 
Director Indicator (EADI). This nonstandard ND control panel was located on the aisle stand just aft of 
the throttles. The ND control panel contained a push-switch that activates the ATAAS system. 

3.1.2 . ATC Station 

Air Traffic Control (ATC) communications were provided to the IFD during the experiment from 
a station located remotely from the simulator cab. The station had a display of air traffic and other 
information so that a single pseudo-controller could provide the real-time communications with other 
simulated traffic and the IFD cab. Pilots’ headsets were used in the simulator cab to simulate radio 
communications. 

A realistic communications environment was created through the activation of prerecorded sound files, 
which simulated transmissions from the other aircraft in the terminal area. The pseudo-controller could 
activate the appropriate sound file after issuing a clearance to an aircraft to simulate a response from that 
aircraft. Communications with the subject aircraft (simulator) were scripted, and the same spacing inter- 
val was used for all runs. 

3.1.3. Eyetr acker 

Pilot gaze and eye movement data were recorded by using an eyetracker (ISCAN Model AA-ETL-500 
low-level infrared, eye-tracking system and supporting software). The eyetracker weighed less than 8 oz 
and was mounted on a baseball cap. The wiring was bundled with the pilot’s headset so that it did not 
interfere with the normal range of pilot head movement. Samples were obtained at 30 Hz. Fixations 
having a minimum duration of 100 ms within a 1-in. square area were recorded. A view of the instrument 
panel and windows and all the instruments and other areas of interest for eyetracker data were recorded 
on videotapes and through the eyetracker data collection software. 

3.2. ATAAS Interface 

Pilots obtained ATAAS guidance from the EADI and ND displays and the command airspeed bug on 
the airspeed indicator. Additional ATAAS status data and crew inputs were provided on Flight Manage- 
ment Computer (FMC) CDU pages. The ATAAS symbology on the EADI and ND appeared only after a 
lead aircraft and spacing interval were selected from the CDU page. 

3.2.1. EADI Display 

A conventional B-757 EADI display format, with a fast/slow (F/S) pointer and scale on the left side of 
the display format (fig. 2) was used for this experiment. Modifications were made on the F/S pointer and 
scale to provide ATAAS guidance to the crew. During the experiment, when ATAAS commands were 
not being generated for the crew, the F/S pointer and scale worked in the conventional manner by indi- 
cating a ± 10-knot airspeed variance from the airspeed selected on the MCP and shown on the electrome- 
chanical Mach/airspeed indicator with the command airspeed bug (fig. 3). 

When ATAAS guidance was provided to the crew, the F/S pointer and scale showed a ± 10-knot speed 
variance from speeds commanded by the ATAAS algorithms. The pointer symbol format was changed to 
a green pentagon with the letter “P” in the middle to clearly indicate to the crew that it was driven relative 
to the ATAAS speed commands (fig. 4). Actual instantaneous ATAAS speed commands were displayed 
to the crew with the command airspeed bug on the Mach/airspeed indicator. 
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Figure 2. EADI with normal symbology (no ATAAS). 
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Figure 4. EADI with ATAAS symbology. 
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A green digital indication of the ATAAS target airspeed was displayed just above the F/S pointer and 
scale. This target airspeed is the ATAAS commanded airspeed when large airspeed changes are not 
required. When large speed changes (10 knots or more) are required, the target airspeed is the final, or 
end-point airspeed that will be commanded. For example, when ATAAS commands a speed reduction 
from 215 knots to 170 knots, the speed annunciated above the F/S indicator will change instantly from 
“215” to “170” to inform the pilot of the speed reduction endpoint. The pointer on the F/S indicator and 
the command airspeed bug on the Mach/airspeed indicator both follow the scheduled gradual speed 
reduction. 

When the ATAAS target airspeed changed, the new commanded speed appeared with a box around it 
that flashed for approximately 5 s to alert the crew. The speed mode annunciation changed to “PDA” 
when the ATAAS guidance was provided. When transitioning to a final approach, the mode annunciation 
changed to a green “APPR” to indicate that the ATAAS algorithm is providing a scheduled speed reduc- 
tion to the final approach speed selected by the flight crew and is no longer trying to maintain separation 
from the leading aircraft. 

3.2.2. Navigation Display 

The ND display was used in a conventional B-757 track-up mode format as shown in figure 5. Other 
traffic was added and appeared on the ND as Traffic Alert/Collision Avoidance System (TCAS) targets 
by using symbology very similar to what pilots use in current operations. 

Figure 6 shows the ND format with symbology added when the ATAAS algorithm was active. 
ATAAS provides three main elements of information: (1) a data block that includes currently entered 
ATAAS data and lead aircraft range, (2) a spacing position indicator, and (3) lead aircraft history dots. 



Figure 5. ND with traffic only displayed (no ATAAS symbology). 


8 



ATAAS 

command speed 


Spacing interval 
entered on CDU 


ID of selected 
lead traffic — 


Current lead 
aircraft range 


Ownship 
position 
symbol ~ 



Lead traffic 
position symbol 


Lead aircraft 
history dots 


Spacing position 
indicator 


Figure 6. ND with ATAAS symbology. 

The data block shows the commanded speed as the top line on the data block. This commanded speed 
is identical to the one that appears above the F/S pointer and scale on the EADI. Below the command 
speed in the ND data block is the spacing interval between ownship and the lead aircraft (in seconds) the 
call sign of lead aircraft, and the current along-track distance in nmi to the lead aircraft. The desired 
spacing interval (120 s in this example) and the call sign of the lead aircraft (UAL 903) are selected by the 
crew through the CDU. The data block updated automatically as the distance between the aircraft changed 
and also reflected any other changes (such as whether the pilots selected a new lead aircraft or spacing 
interval). 

A spacing position indicator shows the position where the ownship should be to achieve the proper 
spacing interval (based on currently entered lead aircraft and desired spacing interval). This indicator is a 
short, green line perpendicular to the ownship ’s ground track, with an inverted “V” attached to the 
midpoint of the line. This indicator provides a visual reference of the ownship’s position relative to the 
desired spacing interval. When the ownship is properly spaced, the spacing position indicator fits over the 
apex of the white triangular ownship symbol. If the spacing position indicator is behind the apex of the 
ownship symbol, the ownship is ahead of where it should be. Conversely, if the spacing position indicator 
is ahead of the ownship symbol, the ownship is behind where it should be. 

The lead aircraft history dots (a trail of white dots extending behind the lead aircraft symbol) show the 
ground track of the currently selected lead aircraft. When an aircraft is initially selected as the lead, its 
history dots appear and instantly extend backwards past the ownship’s current position, depicting the lead 
aircraft’s ground track. This feature allows an ATAAS-equipped aircraft to maintain spacing behind an 
aircraft that is not on an RNAV route. 
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3.2.3. FMC-CDU Pages 


Interface with the ATAAS system takes place through customized FMC-CDU pages, accessed through 
a function key on the CDU, labeled “ATC.” When this key is pushed, the Approach Spacing page appears 
with a list of the other aircraft in the terminal area on the right side of the page, in alphabetical order by 
call sign (fig. 7). 



^ APPR SPACING 1/2 ^ 



SELECT LEAD 


□ 

<SELECT OFF AAL846> 

□ 

□ 

AAL94 1> 

□ 

n 

COA281> 

n 

□ 

UAL225> 

□ 

□ 

UAL903> 

□ 

□ 

APPROACH DATA> 

□ 


V / 


Figure 7. CDU Approach Spacing page before lead aircraft selection. 

When the pilot line-selects the call sign for the lead aircraft, a block of data appears on the left side of 
the page showing the current actual spacing interval, the current actual distance between the lead aircraft 
and ownship, and the lead aircraft’s groundspeed. These data provide the crew a method to check the 
actual spacing against what was assigned, ensuring upon initialization of the system that there was agree- 
ment within some reasonable level (±20 s of assigned interval). A prompt for entering the desired spacing 
interval will appear on Line Select Key 2 Right (LSK2R) on this page. Figure 8 shows that a 90-s spacing 
interval (for ownship to space 90 s behind the lead aircraft, UAL903) has been entered by the ownship 
crew. 
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CURRENT DISTANCE 
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Figure 8. CDU Approach Spacing page after entering lead aircraft and time interval. 


At the bottom of the page shown in figure 8 is a prompt on LSK6R that is labeled APPROACH 
DATA. Pushing this key displays the Approach Data page (fig. 9), which contains prompts for entering 
other relevant parameters, including final approach speed (FAS) for the ownship and the selected lead 
aircraft, minimum distance (if something other than wake-vortex separation is desired), and airport sur- 
face winds. For this experiment, data entry was only required for the FASs because the simulated envi- 
ronment was a calm day, and the standard separation was used. The values for FASs entered on this page 
are especially important on final approach when differences in speeds for the two aircraft could have a 
major effect on the spacing interval achieved at the runway threshold. The value for ownship FAS is the 
target speed used by the ATAAS algorithm when it transitions to approach mode. 
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Figure 9. CDU Approach Data page after entering FASs. 


3.3. Test Scenario 

A single subject pilot was used for data collection, with a confederate pilot (member of experiment 
team) flying in the right seat. The confederate pilot was a retired airline pilot from a major air carrier and 
had previous experience as a participant in research studies at LaRC. Because crew interactions were not 
a focus of this study, this crew arrangement provided the opportunity to obtain data on acceptability and 
workload from the subject pilot while still maintaining the realism of operating in a two-person crew, full- 
mission environment. 

The simulated environment for this study was the Memphis International Airport (MEM) and sur- 
rounding terminal area. Calm wind conditions and visibility of 10 mi in haze were simulated. The traffic 
level corresponded to what might be expected at a busy terminal area. Normal radio communications with 
ATC were provided. Other traffic in the pattern was generated from prerecorded tracks and was shown on 
the ownship displays and the out-the-window computer-generated imagery system. 

The routing flown for the scenario was a modified WLDER4 STAR (fig. 10). Modifications to the 
STAR include a published downwind and base leg routing for a transition to the Instrument Landing 
System (ISL) runway 36 Right final approach course. The same flight scenario was used for all runs and 
began with the aircraft level at 8000 ft, 250-knot indicated airspeed (IAS), approximately 10 nmi prior to 
the downwind turn. 

A single stream of arriving traffic was simulated and used for all runs (i.e., call signs of the aircraft in 
the arrival stream were the same for all runs). The subject aircraft (call sign “NASA 557”) was number 8 
in the trail to the runway at the start of the scenario. The traffic aircraft (call sign UAL903) immediately 
ahead of NASA 557 and all other aircraft in the scenario followed the nominal charted speeds in an 
orderly manner, with no unusual or rapid changes in speed. The approach spacing clearance was issued 
after the aircraft had turned onto the downwind leg and switched the communication radios to the 
approach control frequency. The normal approach clearance was separate from the approach spacing 
clearance and was issued when the aircraft was on base leg. The assigned lead aircraft (traffic to follow) 
and spacing interval in the clearance were the same for all runs. 
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Figure 10. Modified WLDER4 arrival, Memphis International Airport. 


3.3.1. Test Setup 

Each test subject pilot flew both as a Pilot Flying (PF) and as a Pilot Not Flying (PNF) during four dif- 
ferent test conditions, as shown in table 1 . 


Table 1. Test Matrix 


Test condition 

Pilot position 

Pilot 

flying 

Pilot not 
flying 

Baseline 

1 

5 

Manual throttle 

2 

6 

MCP speed 

3 

7 

Automatic 

4 

8 


Baseline test runs were flown as a conventional descent and approach without ATAAS speed com- 
mands. These test runs provided the test subjects a comparison for workload and acceptability ratings 
with the other test runs, in which ATAAS provided airspeed commands. 

The Manual Throttle, MCP Speed, and Automatic test conditions were flown by using ATAAS- 
computed airspeed commands. During the Manual Throttle test runs, the PF controlled the thrust levers 
manually to maintain airspeed. During the MCP Speed runs, the autothrottles were engaged and the PF 
selected airspeeds (to be flown) through the MCP panel. During the Automatic test runs, the PF moni- 
tored airspeed commands to the autothrottles that were automatically inserted by the ATAAS algorithm. 
The autopilot was engaged during all these and the baseline test runs. 
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Eight different pilots participated in this experiment. Each pilot was scheduled to complete all brief- 
ing, training, and testing in one day. The morning started with a verbal briefing to supplement a copy of 
the flight manual bulletin and charts, which were previously mailed to the subject pilot. Included in the 
bulletin was background on the operation and the charted procedure, a summary of the procedures for 
interacting with the custom ATAAS FMC CDU pages, and a checklist indicating the crew member 
responsibilities for each step in the operation (see appendix A). All data runs were completed in the 
afternoon. 

During the simulator training, each pilot received a short verbal briefing by the confederate pilot that 
covered differences between the simulator cab and a typical B-757 aircraft. Each test subject then flew 
three complete runs for training, one with each of the different methods of speed control used for follow- 
ing the ATAAS guidance. For the last of the three training runs, the subject pilot was fitted with the 
eyetracker equipment and a radio headset to demonstrate what the data runs would be like with radio 
communications and to obtain initial eyetracker calibration data. 

Each pilot completed a test matrix of eight data runs (table 2). Each data run lasted approximately 
20 min. A Latin Square design was used to order the runs to minimize the potential for interactions of the 
test variables. Calibration of the eyetracker and completion of a workload rating and a questionnaire 
following each run took approximately 10 min. Following completion of the last run, the subject pilot was 
taken back to the briefing room to complete the post-test questionnaire. 


Table 2. Ordering of Runs for All Pilots 


Condition 

number 

Pilot 1 
order 

Pilot 2 
order 

Pilot 3 
order 

Pilot 4 
order 

Pilot 5 
order 

Pilot 6 
order 

Pilot 7 
order 

Pilot 8 
order 

1 

1 

6 

8 

3 

7 

5 

2 

4 

2 

5 

2 

4 

1 

6 

8 

3 

7 

3 

2 

4 

1 

6 

8 

3 

7 

5 

4 

3 

7 

5 

2 

4 

1 

6 

8 

5 

6 

8 

3 

7 

5 

2 

4 

1 

6 

8 

3 

7 

5 

2 

4 

1 

6 

7 

4 

1 

6 

8 

3 

7 

5 

2 

8 

7 

5 

2 

4 

1 

6 

8 

3 


3.3.2. Crew Procedures 

During each run, the PNF was responsible for making inputs to the flight management system through 
the CDU, which included selecting the assigned traffic to follow, entering the assigned spacing interval, 
and any other necessary data (such as final approach speeds) on the ATAAS CDU pages. The PNF also 
acknowledged the clearance with ATC. The PF was responsible for activating the ATAAS system and 
following the speed commands. Both pilots were responsible for monitoring speed and other cues to 
ensure compliance with the speed commands. However, during this test, the confederate pilot did not 
advise the subject pilot when to adjust speed. These tasks were integrated with other normal duties. Each 
subject pilot acted as the PF in half the runs he completed and as the PNF in the other half. 

The autopilot was engaged during all test runs. The autothrottles were engaged on all test runs except 
the ones in which manual throttle operations were required. All the ATAAS runs consisted of complete 
Flight Management System (FMS) routes that were flown in the lateral navigation (LNAV) mode for 
lateral guidance. The baseline runs were flown as they are currently flown in real-world operations, with 
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LNAV for lateral guidance until the end of the STAR, and then transitioning to the HEADING SELECT 
mode to comply with vectors from ATC. The pitch axis and throttle are integrated and included Altitude 
Hold (pitch controls altitude) and Speed (thrust controls airspeed) during level flight, and Speed (pitch 
controls airspeed) and Flight Level Change (thrust fixed at a constant setting, generally idle thrust) during 
descents. The Vertical Navigation (VNAV) flight mode was not used in this test. The descents were flown 
in Flight Level Change (FLCH) mode. 

3.3.3. Subject Pilots 


Eight subject pilots were used for data collection. The pilots were required to be type-rated and current 
in the B-757 aircraft. Total flight time for each pilot ranged between 4000 and 17 000 h. Two pilots had 
between 300-1000 h in type, and the remainder had over 1000 h in type. There were five first officers 
and three captains, from four different airlines. A summary of the subject pilot’s experience is shown in 
table 3. 


Table 3. Summary of Subject Pilot Flying Experience 


Pilot 

Total flight hours 

Hours in type (B757) 

Current airline 

A 

6 200 

>1000 

CO (FO) 

B 

8 100 

300-1000 

AAL (FO) 

C 

10 000 

>1000 

UAL (C) 

D 

4 000 

>1000 

AAL (FO) 

E 

10 000 

>1000 

UAL (FO) 

F 

10 226 

>1000 

AAL (C) 

G 

4 000 

300-1000 

UAL (FO) 

H 

17 000 

>1000 

NWA (C) 


3.3.4. Data Collection 

Recorded data included aircraft state data from the simulator, data from the ATAAS system, and eye 
point-of-regard data from an eyetracker and headtracker. Aircraft state data included position, altitude 
time histories, and autoflight system mode. Data from the ATAAS system included the state in which the 
system was operating (i.e., armed, engaged, approach), in addition to the commanded speed and the time 
interval and distance between the ownship and lead aircraft. Eye point-of-regard data consisted of many 
positional parameters, which were used after the test to compute the two variables of primary importance: 
the dwell time (how long the pilot looked at a particular field of view each time he glanced at it) and the 
allocation of time on particular fields of view (i.e., for the duration of each test run and how the pilot’s 
attention was divided among the different fields of view, which are referred to as Areas of Interest (AOI). 

Subjective data were obtained through questionnaires to solicit the pilots’ opinions on various issues 
related to using the ATAAS tool. Two structured questionnaires were administered, one following com- 
pletion of each of the ATAAS runs (the Post-Run Questionnaire), and one at the end of the day, after 
completing all runs (the Post-Test Questionnaire). Using the NASA Task-Load Index (TLX) method 
(ref. 6), the pilots completed a workload self-assessment after each run. Copies of the questionnaires are 
included in appendix B, and a copy of the TLX is shown in appendix C. 

4.0. Results 

One goal of this study was to validate the results from the previous Monte Carlo analyses, which 
evaluated analytically the ATAAS algorithm’s ability to provide appropriate speed guidance to deliver the 
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aircraft at the runway with the proper spacing interval. In implementing the algorithm in the simulator, the 
system’s performance becomes more dependent on the manner in which the guidance is followed and 
could have an effect on the spacing interval ultimately achieved. It is this difference, based on the method 
used for controlling the ATAAS speed to follow the ATAAS commands, that was explored in the data 
analysis for this study. 

In the baseline scenario, the goal was not to try to achieve the same or similar results for interarrival 
spacing as in the ATAAS conditions because the controller’s ability to space aircraft was not relevant. 
The baseline scenario was flown to provide the pilots a comparison for workload and acceptability ratings 
but was not a valid comparison for arrival time intervals because the traffic data were prerecorded and an 
actual facility controller was not used. Spacing intervals for the baseline runs were not computed. To 
make a valid comparison of spacing intervals achievable by a controller versus ATAAS, a separate test 
would be needed, with experimental controls appropriate to the different objectives of that test. 

4.1. Arrival Time Interval 

The time -based spacing interval between the lead aircraft and the subject aircraft was compared for all 
test runs in which ATAAS was used. This interval is the elapsed time from when the lead aircraft crosses 
the runway threshold until the following aircraft (the following aircraft is being flown by the subject 
pilots) crosses the threshold. 

Figure 11 shows the mean and standard deviation of the time-based spacing interval at the runway 
threshold for all ATAAS runs. The results are presented by the method used for speed control as follows: 
Manual (pilot controlled speed by manually moving throttle levers, with auto throttles off); MCP (pilot 
controlled speed from MCP speed window); and Automatic (speed controlled by autothrottles coupled 
with ATAAS speed commands). The target spacing interval was 90 s for all runs. The Automatic 
condition resulted in the most accurate spacing, with mean intervals within 1 s of the target (standard 
deviation = 1.7) for both the PF and PNF positions. These results compare well with the Monte Carlo 
results, in which the mean error was slightly over 1 s, with a standard deviation of 2.49 s. 
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Figure 1 1 . Spacing interval at mnway threshold. 
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The lower end of the standard deviation is below 90 s, meaning that in some of these cases, the spac- 
ing interval at the runway was less than the assigned 90-s interval. Since the responsibility for maintain- 
ing a safe interval between the aircraft rests with the air traffic controller, the pilot is not required to 
initiate a go-around if the spacing interval falls below the assigned interval, as long as he continues to 
follow the ATAAS command speeds. With this implementation of the algorithm, the pilot did not have 
any indication of what the actual spacing interval was when ownship crossed the threshold because this 
information ceased to be displayed after the ATAAS algorithm transitioned to its approach mode. 

The mean spacing interval errors for the Manual and MCP conditions were 4 s and 5 s late, respec- 
tively. Standard deviations for these conditions were comparable with the Automatic condition, both for 
PF and PNF positions. These results are consistent with expectations. In the Automatic condition, the 
aircraft is being controlled directly by the ATAAS speed command output, so it is expected that it would 
result in an interval close to the target. In the other two cases, the ATAAS algorithm is providing the 
same commands, but rather than a direct connection with the autothrottles, the aircraft speed is being 
controlled manually by the pilot. In these cases, the spacing interval error is subject to how closely the 
pilot controls the speed relative to the command speed. 

This speed tracking performance is especially crucial during the final deceleration (in the final ap- 
proach segment) and is the most significant factor in the performance differences among the three speed 
modes used in this study. A previous study showed that differences of 10-15 knots in final approach 
speed during the last 2 to 4 miles prior to the final approach fix (FAF) could cause differences on the 
order of 3-5 s (ref. 7) in actual versus predicted arrival time at the FAF. The effect that this type of error 
can have on the threshold arrival time cannot be corrected after the trailing aircraft has decelerated to FAS 
because the ATAAS algorithm is no longer computing a spacing interval from the lead aircraft. 

The net effect of the pilots slowing down quicker than the scheduled ATAAS deceleration is a result- 
ing spacing interval that is a few seconds longer than the target interval, as seen in figure 1 1 . This case 
occurred in the Manual and MCP conditions, as shown in figure 1 1, as a result of the pilots’ responses to 
the annunciated speed commands and could be mitigated by training or display changes that would 
encourage the pilots to follow the ATAAS speed guidance more closely. In figure 11, the higher standard 
deviation in the Automatic condition (with the subject pilot acting as PNF) was caused by a failure to 
enter the correct final approach speed for the lead aircraft, which resulted in a higher than normal spacing 
error at the threshold. 

The ATAAS algorithm provided a means for achieving a target threshold interarrival time interval 
within a mean error of ±5 s, which equates to approximately 1 100 ft at the approach speed of 130 knots. 
Whether or not this performance level is adequate most likely depends on the specific traffic conditions 
and what level of throughput a facility is trying to achieve. In this study, it was found that the best results 
(±1 s, equivalent to 220 ft) were achieved with closely controlled speed (i.e., speed was carefully 
controlled to precisely follow the ATAAS command speed), which was achievable with the ATAAS 
Autothrottle-coupled (the Automatic condition). 

4.2. Pilot Interaction With Guidance 

The number of times during each run that the ATAAS command speed changed was used as a factor 
to evaluate pilot acceptability, which gave an indication of the additional workload required when work- 
ing with the ATAAS system. The route segment in which the number of speed changes was assessed 
started on the downwind leg where ownship was at 210 knots, descending from 8000 ft to 3000 ft. This 
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Table 4. Number of Commanded Speed Changes 



Mean 

Standard deviation 

PF Manual 

7.3 

1.3 

PF MCP 

7.5 

1.5 

PF Automatic 

5.4 

0.6 

PNF Manual 

6.4 

1.3 

PNF MCP 

6.6 

1.7 

PNF Automatic 

5.0 

0.4 


path segment ended at the final approach fix, where the ownship altitude was about 2300 ft mean sea 
level (MSL) and final approach speed was typically 135 knots (landing gear and flaps would be in a 
landing configuration). 

The data summarized in table 4 reflect the number of times that the annunciated ATAAS speed com- 
mand changed throughout the duration of each run. These data were obtained from videotape recordings 
of the primary instruments (EADI and ND) during the data runs. In a current-day type operation (using 
the Memphis arrival traffic area as an example), ATC would typically issue two or more speed reductions 
during this route segment plus the pilot-initiated deceleration to the final approach speed at the final 
approach fix. 

The pilots in this study made more speed changes with ATAAS than are typically seen in the current- 
day environment. However, by making more frequent, but smaller speed changes, the more accurate an 
interarrival runway threshold spacing interval can be. Because the duration of each run was about 
20 min, a change occurred, on average, about once every 2.5 to 3 min. 

Comparing the conditions in table 4 across PF and PNF roles also shows that for the MCP and Manual 
conditions, when the subject pilot acted as PNF, there were fewer required changes in commanded speed 
observed. This result could be attributed to the confederate pilot’s following the command speed more 
closely than did the subject pilots because the confederate pilot knew to more slowly follow the changes 
in the guidance. The confederate pilot had the same initial training as the subject pilots but had about 
2 weeks more experience using the ATAAS system. The confederate pilot’s higher performance level 
might be expected after having used the ATAAS system on a more frequent basis for a period of 
2 weeks; however, a larger sample of data would be needed to more accurately quantify this effect. 

The data in table 4 indicate that there were more speed changes required in the MCP and Manual con- 
ditions than in the Automatic condition. This result is not surprising because the autothrottles incremen- 
tally changed the speed to stay close to the ATAAS speed profile during speed reductions and resulted in 
fewer errors requiring correction. In the MCP and Manual conditions, the manner in which the pilots 
followed the speed guidance resulted in a greater number of required speed changes because of errors 
introduced when the ATAAS speed profile was not followed closely. Typically, these errors occurred 
when ATAAS commanded a large (greater than 10-knot) speed change, and the pilot dialed the MCP 
speed window or pulled the throttles back quickly enough to produce a greater deceleration rate than the 
ATAAS profile stipulates. Because the ATAAS deceleration rate was slower than that used in the simu- 
lator performance model (to make the ATAAS algorithm compatible with most aircraft types), the result 
was a steeper deceleration profile than the algorithm expected and often required a subsequent correction 
of about 5 knots. This effect can be seen by comparing data for a typical MCP condition run and a typical 
Automatic condition run. 
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Indicated Airspeeds 



900 800 700 600 500 400 300 200 100 0 

Ownship time to threshold, s 


Figure 12(a). Time histories of ATAAS commanded speed, ownship indicated airspeed, and MCP window speed 
for MCP condition. 


ATAAS Goal Time Error 



Ownship time to threshold, s 


Figure 12(b). Accumulated time error (ATAAS predicted interval minus actual interval) for MCP condition. 

Figure 12(a) shows time histories of three variables for an MCP test run: the ATAAS commanded 
speed, the ownship ’s actual indicated airspeed, and the speed dialed into the MCP window. Figure 12(b) 
shows the cumulative time error for the same data run. The time error is essentially the target spacing 
interval minus the current spacing interval. Figures 13(a) and (b) show the same plots for an autothrottle 
test run. The plotted data begin at the time the first valid commanded speed was displayed and end after 
the algorithm has transitioned to its Approach mode (and is no longer computing a spacing interval). 
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Figure 13(a). Time histories of ATAAS commanded speed, ownship indicated airspeed, and MCP window speed 
for autothrottle-coupled condition. 
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Figure 13(b). Accumulated time error (ATAAS predicted interval minus actual interval) for autothrottle-coupled 
condition. 

At about t = 510 s, the ATAAS algorithm commanded a speed change from 215 knots to 170 knots. 
The deceleration profile used by the ATAAS algorithm shows a deceleration lasting about 100 s. The 
pilot dialed the MCP window speed quickly down to 170 knots, which initiated a deceleration that was 
faster than the ATAAS profile, as shown by the solid black line (ownship IAS). This deceleration lasted 
only about 50 s, leaving the ownship at the target speed much more quickly than predicted by the ATAAS 
algorithm. To compensate, the ATAAS algorithm then commanded another speed change back up to 
180 knots for about 20 s (at t = 420) before finally returning to the original commanded speed of 
170 knots. All pilots executed the speed changes in the MCP test runs in a similar manner. 

This practice is typical of what most pilots do in the current-day environment. When ATC issues a 
speed change, the pilot typically dials the MCP window speed quickly to the assigned speed, letting the 
aircraft slow at its own rate, rather than trying to follow a deceleration schedule. To avoid the need for 


19 


corrections (as illustrated in this example) would require the pilots to be able to follow the ATAAS com- 
mand speed more closely. This ability would require changes to the ATAAS display elements that would 
allow the pilot to follow the ATAAS deceleration profile or to receive additional training for the pilots on 
what the deceleration profile used by the ATAAS algorithm looks like. 

The effect on the spacing interval can be seen clearly in figure 12(b), as the error is at about -3 s and is 
approaching zero at t = 520 s (when the speed change is initiated) but then begins to depart again as the 
aircraft slows down more quickly than predicted by the ATAAS algorithm. After the ATAAS command 
speed correction, the error again begins to dissipate towards zero. This effect is seen again on a smaller 
scale just before the lead aircraft crosses the threshold when the ownship begins to slow to approach 
speed. The large error at the start of the data is a transient from the initialization of the algorithm and was 
evident in all the data runs. This transient did not affect the spacing interval results because it dissipated 
with plenty of time to allow fine-tuning of the spacing interval, regardless of the method used for speed 
control. 

In figure 13(a), the ATAAS command speed profile is seen leading the ownship IAS by a few sec- 
onds, showing the close coupling between the ATAAS deceleration profile and the ownship speed when 
the autothrottle guidance follows the ATAAS command speed (in the Automatic test runs). This type of 
behavior was seen in all Automatic test runs. Because the speed control task did not involve interaction 
from the pilots, there were no pilot-specific differences. The effect on spacing interval error can be seen 
in figure 13(b), in which the time error remains within ±2 s throughout most of the latter half of the run. 

4.3. Eyetr acker Results 

An eyetracker recorded the subject pilots’ eye movements to ascertain that the introduction of the 
ATAAS tool on the flight deck was not detrimental to the pilots’ out-the-window scan. It was noted 
during the data collection that all pilots attempted to see the other traffic that was visible out the window. 
The eye movement results of this study address issues related to the effect of ATAAS on pilot visual 
attention and can offer objective support for pilot judgments of ATAAS acceptability under the varying 
levels of automation. The complete analysis results are published separately (ref. 8). A summary of the 
results pertaining to differences in out-the-window scan attributed to the use of ATAAS are presented in 
this section. 

The dwell or duration of fixations was defined as the time between entering and exiting an area of 
interest (AOI). The following AOIs were defined, as shown in figure 14: (1) EADI, (2) ND, (3) Airspeed 
Indicator, (4) Altimeter, (5) MCP, (6) Window, (7) Left side CDU, (8) instruments on the right side, and 
(9) Right side CDU. 

4.3.1. Eyetracker Data Analysis 

Prior to analysis, data were segmented through review of the eyetracker videotapes into sets labeled 
“Downwind” and “Final Approach.” The Downwind segment comprised the period from the start of the 
run to completion of the turn onto the base leg of the arrival pattern (but not including the full base leg). 
Final Approach started with transmission of the Final Approach clearance as the aircraft began to turn 
onto the Final Approach leg and concluded with touchdown. 
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Figure 14. Definition of eyetracker AOIs. 


4.3.2. Pilot Scan Pattern 

The effect of ATAAS on pilots’ visual scan patterns was examined through a link analysis, a method 
of assessing the pattern of how a person’s gaze transitions from one area of interest to another, such as 
from the EADI to the airspeed indicator. Link values represent the percentage of unidirectional eye 
movements between defined AOIs (i.e., movement from one AOI to another). The link analysis was 
conducted separately on the Downwind and Final Approach data sets. Comparisons were made among all 
ATAAS conditions and the Baseline condition, and also between each of the ATAAS conditions to 
examine the effect of ATAAS in conjunction with the different methods of speed control. 

Overall, the pilots’ scans did not appear to exhibit a definable sequence of eye movements from one 
AOI to another because the link values were nearly equivalent in either direction — equally true of both 
ATAAS and baseline conditions. The strongest link in nearly all conditions was from EADI to Airspeed, 
which was expected, since the additional task required with the use of ATAAS is to follow the airspeed. 
This increase was higher in all ATAAS conditions than in the comparable (PF or PNF) baseline condi- 
tions for both the Downwind and Final Approach flight segments. However, the difference was only 1 or 
2 percent, which is not a significant change. The introduction of ATAAS did not result in any pilot eye 
scans between instruments that were different from those recorded for the Baseline conditions. 

4.3.3. Allocation of Visual Attention 

The pilots’ allocation of visual attention is inferred from the proportional distribution of dwell time, 
that is, the proportion of simulation run time in each flight segment (Downwind or Final Approach) that 
the pilot’s gaze remained within an area of interest. An 8x8 within subjects Analysis of Variance 
(ANOVA) was conducted on percent dwell time by Condition, Subject, and Run for each of the seven 
AOIs. For this study, the AOI of prime concern was what effect the ATAAS command and pilot tasks 
have upon the out-the-window eye scan. 
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An analysis was conducted to determine whether there were statistically significant differences in the 
effects upon the out-the-window scan caused by the differences in the test conditions, differences among 
the pilots, and the order in which the test runs were conducted. In the downwind segment, no significant 
effects were found for the allocation of attention to the window AOI, although there were significant 
effects for other AOIs. For the Final Approach segment, the ANOVA conducted on proportion of time 
allocated to the Window AOI showed significant differences among test conditions. The amount 
of attention given to the Window was significantly higher in PF Auto (18 percent) than in PF Manual 
(10 percent), which was expected, because the pilot has more time to look out the window when he does 
not have to manually control the speed. 

4.3.4 . Dwell Time 

Dwell time is the length of time that the pilot’s gaze remained within an AOI without moving outside 
of that area. The analysis performed on dwell time was the same as in the preceding section for allocation. 

The mean dwell time for pilots looking out the window was greater on the final approach segment 
than on downwind for all test conditions. This result was expected because the pilots had visual reference 
to the runway on which they were landing. For the downwind segment, the Manual and MCP conditions 
showed lower mean dwell times than the Baseline condition, and Automatic showed higher mean dwell 
times, meaning that the pilots had less time to dwell on looking out the window during the Manual and 
MCP conditions than during the Baseline condition, and more time during the Automatic condition than 
during the Baseline condition. However, on final approach, a different effect is seen, with longer dwell 
times for the Manual and MCP conditions than Baseline and slightly higher dwell times for the Automatic 
condition versus Baseline. The dwell times for the Automatic condition are very similar to the Baseline 
condition, with slightly higher variation. 

4.3.5. Summary of Eyetr acker Results 

The effects of the ATAAS guidance were analyzed through comparisons between the MCP and Base- 
line conditions because for these two conditions, speed changes were made in the same manner (the pilot 
dialed the desired speed into the MCP speed window). Few differences in visual attention were found that 
could clearly be attributed to ATAAS. The only statistically significant effect was that the percentage 
allocation of attention to the MCP was significantly greater (by 4 percent) in PF MCP than in PF Baseline 
during the Final Approach flight segment. This effect was most likely due to a greater number of speed 
changes during this segment on ATAAS runs versus baseline runs. No significant differences in dwell 
time were found in comparisons between the PF MCP or PNF MCP and the comparable Baseline condi- 
tions in either the Downwind or Final Approach data sets, indicating that the pilots’ scans were not 
changed significantly by the addition of the ATAAS tool. Also, few differences in pilot scan patterns 
were evident. Small decreases in eye movements from all instruments to the Window can also be attrib- 
uted to ATAAS (indicating that less time was available to spend looking out the window), but none of the 
individual links decreased by more than 1 percent. In all, ATAAS reduced eye movements to the Window 
by 3 percent during Downwind and by 2 percent during Final Approach. 

4.4. Workload Results 

4.4.1. TLX Workload Estimates 

The NASA Task-Load Index (TLX) method (ref. 6) was used to obtain a perceived workload rating 
from each test subject after each test run. The NASA TLX is a multidimensional workload rating scale 
that uses six different dimensions to assess a test subject’s perceived workload when performing a given 
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task. For the current study, each dimension of the TLX rating was scored on a scale with a range from 0 
(low difficulty/demand) to 20 (high difficulty/demand). The six TLX scales used for rating the workload 
(Mental Demand, Physical Demand, Temporal Demand, Effort, Performance, and Frustration) are 
described in appendix D. The TLX measure was a composite created by averaging the six TLX scale 
values for each participant. The composite score was used because the individual TLX scales were highly 
correlated with one another and thus provided redundant information. A 2 x 4 [(Pilot Position: Flying, 
Not Flying) by (Condition: Baseline, Automatic, MCP, Manual)] within subjects ANOVA on the TLX 
workload estimates showed a significant difference in reported workload for pilot position, F(l, 7) = 7.80, 
p < .05. As expected, flying the aircraft (Mean (M) = 14.68, Standard Deviation (SD) = 10.55) elicited 
higher workload estimates than not flying the aircraft (M = 9.60, SD = 6.04). No significant differences 
were found for condition, F(3, 21) = 1.82, p > .05. Thus, none of the methods of speed control that were 
used differed significantly from the baseline workload estimates or from one another (see fig. 15). This 
result indicates that the pilots did not perceive a significant increase in workload while using the ATAAS 
system compared with standard procedures. The speed control methods used for following the ATAAS 
commands covered three different levels of pilot involvement in adjusting aircraft speed. Even with the 
additional piloting task required to follow the ATAAS commands, plus the additional CDU interface 
required for setting up the ATAAS system, the workload level perceived by the pilots did not show 
statistically significant differences from baseline conditions. 
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Figure 15. Mean TLX workload estimates (PF) for each condition. 

4.4.2. Post-Run Workload Questionnaire 

The post-run questionnaire had the participants compare their level of mental, physical, and overall 
workload produced by the task to the workload level for a standard (non-ATAAS) approach. The 
responses for these scales correlated very highly with one another (table 5) and were combined into a 


Table 5. Pearson Correlations Among Post-Run Workload Comparisons 



Physical 

Mental 

Overall 

Physical 


.71* 

.82* 

Mental 

.71* 


.86* 

Overall 

.82* 

.86* 



*p < .05 
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Figure 16. Mean post-run workload comparisons for mode as function of pilot position. 


single composite workload measure by averaging the three scores for each participant. A 2 x 3 [(Pilot 
Position: Flying, Not Flying) by (Mode: Automatic, MCP, Manual)] within subjects ANOVA on the 
composite workload scores showed no significant differences for any of the effects. Because the ques- 
tionnaire asked the pilots to compare the ATAAS conditions with the Baseline, the questionnaire was not 
administered for the Baseline conditions. 

Figure 16 illustrates the post-run composite workload comparisons for the experimental conditions. 
The scores were adjusted to illustrate perceived workload comparisons relative to nominal ILS approach 
procedures. Positive values for an experimental condition indicate higher workload than a nominal ILS 
approach procedure, while negative values indicate lower workload (values for the comparisons were 
between -3 and 3). Note that the original scale was from 1 to 7; the adjustment to a scale of -3 to +3 was 
done for better visualization of differences in the figure, while preserving a 7-point scale. 

Table 6 shows that the workload estimates given for the TLX appear to correlate well with the work- 
load comparisons given for the post-run questionnaire. These correlations serve as a rough reliability 
check and confirm that the mental and physical scales of the TLX are significantly correlated with the 
mental and physical scales of the post-run questionnaire, respectively. 

Table 6. Pearson Correlations Between TLX Scales and Post-Run Workload Comparisons 



Physical workload 
(post-mn) 

Mental workload 
(post-mn) 

Overall workload 
(post-mn) 

TLX Mental 

.198 

.401* 

.332* 

TLX Physical 

.297* 

.400* 

.294* 

TLX Temporal 

.277 

.350* 

.225 

TLX Performance 

-.148 

.062 

-.114 

TLX Effort 

.335* 

.517* 

.395* 

TLX Fmstration 

.151 

.335* 

.241 


*p < .05 
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4.5. Post-Run Questionnaire 
4.5.1. Overall Acceptability 

Pilot acceptability ratings for the ATAAS tool were examined separately for downwind, base, and 
final approach segments from questions that used a 7-point scale (1 = not at all acceptable, 4 = borderline, 
and 7 = very acceptable). Pilots responded to the question, “How acceptable was the ATAAS tool during 
the following phases of the approach procedure?” 

A separate 2x3 [(Pilot Position: Flying, Not Flying) by (Condition: Automatic, MCP, Manual)] 
within subjects ANOVA was performed on each approach segment and showed no significant differences 
for any of the effects. The means for overall ratings of acceptability for the ATAAS tool, across all 
conditions, were quite high, as can be seen in table 7. These results indicated an overall acceptability of 
the ATAAS tool by the subject pilots. 

Table 7. Mean Overall Acceptability of ATAAS Tool by Flight Segment 
Across All Conditions 


Phase of approach 

Mean 

Standard deviation 

Downwind 

6.38 

.96 

Base 

6.62 

.53 

Final 

6.49 

.66 


4.5.2. Acceptability of Head-Down Time 

The acceptability of the amount of head-down time was examined with the question, “How acceptable 
was the amount of head-down time for the following phases of the approach procedure?” A 2 x 3 [(Pilot 
Position: Flying, Not Flying) by (Condition: Automatic, MCP, Manual)] within subjects ANOVA showed 
no significant differences for any effects for any of the phases of approach (downwind, base, final). 
Subject pilots across all conditions rated the amount of head-down time quite acceptable (table 8). 

Table 8. Mean Acceptability of Head-Down Time Across All Conditions 


Phase of approach 

Mean 

Standard deviation 

Downwind 

6.00 

1.37 

Base 

6.23 

1.07 

Final 

6.38 

.80 


4.5.3. Confidence With the ATAAS Tool 

Confidence ratings with the ATAAS tool’s guidance were examined by using a 7-point scale (1 = not 
at all confident, 4 = borderline, and 7 = very confident). Participants responded to the question, “How 
confident were you with the guidance provided by the ATAAS tool during the following phases of the 
approach procedure?” A 2 x 3 [(Pilot Position: Flying, Not Flying) by (Condition: Automatic, MCP, 
Manual)] within subjects ANOVA on the confidence ratings was performed on each phase of approach 
(downwind, base, and final). No significant effects were found in any of the three ANOVAs. Participants 
across all conditions rated their level of confidence quite high for all phases of the approach (table 9). 
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Table 9. Mean Confidence in Guidance Provided by AT A AS Tool 
Across All Conditions 


Phase of approach 

Mean 

Standard deviation 

Downwind 

6.35 

1.06 

Base 

6.64 

.57 

Final 

6.62 

.58 


The statistical power (a result of the observed power calculated in the ANOVAs performed on the 
workload data) in all of the previously described analyses was too low to detect any true differences 
among the conditions. 

4.6. Post-Experiment Questionnaire 

The Post-Experiment Questionnaire was divided into the following sections: Acceptability, Proce- 
dures, Flight Management Computer-Control/Display Unit, Navigation Display, Electronic Attitude 
Director Indicator, Training, and Comments. Selected questions from each of the sections are included in 
the discussion to give a picture of the overall pilot opinions. The questions that required selecting a rating 
used a seven-point scale, in which the descriptors for the midpoint (4) indicated neutral conditions (i.e., 
borderline, unchanged, or moderate). Descriptors for the high end of the scale (7) indicated the more 
desirable condition (i.e., completely effective, completely comfortable, extremely clear, and very easy), 
and for the low end (1) were the more undesirable conditions (i.e., completely ineffective, completely 
uncomfortable, extremely unclear, and very difficult). Responses to these questions are summarized in 
table 10 as means and standard deviations. The pilots were also free to insert comments to help clarify 
their responses. In addition to these questions, a number of open-ended, essay-type questions were asked. 
The more common or significant pilot responses to these questions are also summarized in the discussion. 

4.6.1. Acceptability 

The pilots gave high ratings to the Approach Spacing concept in all questions (Qs) in the Acceptability 
section. Generally, they answered that they were comfortable using the ATAAS tool to maintain the 
assigned spacing in the terminal area (Question 1), and that it was effective in helping maintain the 
assigned spacing (Q2). The pilots’ perception of level of safety was that it was slightly enhanced when 
using the ATAAS tool over nominal ILS approach procedures (Q3). The pilots also responded that the 
approach spacing procedures and tools could be effectively integrated into the current flight deck opera- 
tional environment (Q7), and that they had confidence in the speed guidance provided by the tool (Q4). 

The subject pilots were also asked if they could think of any real-world situations in which the 
ATAAS procedures and tools might not be effective. The most common responses were in adverse 
weather conditions that required deviations and in gusty wind conditions. Two pilots mentioned that 
variations in speed control of the lead aircraft could possibly cause a ripple effect on all following aircraft, 
and one pilot mentioned the airspace limitations as a potential problem. One pilot also mentioned, as a 
possible problem, a “last minute” insertion of an aircraft into the flow. 

4.6.2. Procedures 

The subject pilots gave high ratings to the overall procedure for using the ATAAS guidance system 
(Q9), as well as the clarity of the phraseology used for the approach spacing clearance (Q1 1). The subject 
pilots indicated how comfortable they were accepting the assigned spacing interval (Q12). When asked to 
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Table 10. Summary of Post-Experiment Questionnaire Responses 


Section 

Question 

Mean 

Standard deviation 

Acceptability 




Qi 

6.19 

.65 

Q2 

6.31 

.46 

Q3 

5.44 

1.40 

Q4 

5.94 

.86 

Q5 

5.56 

.62 

Q6 

5.63 

1.19 

Q7 

6.13 

.83 

Procedures 




Q9 

6.38 

.74 

Q10 

6.38 

.92 

Qll 

6.38 

1.06 

Q12A 

6.50 

.53 

Q13 

3.00 

.93 

FMC-CDU 




Q15 

6.75 

.46 

Q16 

6.75 

.46 

Q17 

6.00 

1.83 

Q18 

6.63 

.52 

Q19 

6.50 

.53 

Q20 

6.38 

.52 

Q21 

6.38 

.74 

Navigation display 




Q24 

5.06 

1.66 

Q26 

6.25 

.71 

Q27 

6.00 

.76 

Q29 

6.00 

.93 

Q30 

6.13 

.64 

EADI 




Q35 

4.50 

1.69 

Q36 

5.75 

1.04 

Q37 

6.13 

.83 

Q38 

6.31 

.80 

Training 




Q41 

6.25 

.71 

Q42 

2.50 

1.77 

Q43 

3.38 

1.60 

Q44 

5.00 

1.85 


compare the amount of time spent looking at displays for the approach spacing procedure with the amount 
of time spent looking at displays with standard procedures, the responses indicated that it was slightly 
higher for the ATAAS approach spacing procedure (Q13). None of the pilots made any comments that an 
unacceptable level of head-down time was required when using the ATAAS commands. 

4.6.3. Flight Management Computer-Control Display Unit (FMC-CDU) 

The subject pilots were asked how well the CDU ATAAS pages provided clear (i.e., unambiguous) 
and consistent labeling, and the responses indicated that they found the information to be very clear 
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(Q16). The methods for selecting the lead aircraft and for entering the spacing interval were also rated 
highly for adequacy (Q18 and Q19, respectively), as was the acceptability of the amount of time spent 
working with the CDU ATAAS pages (Q21). 

When asked which piece of information they would add to the CDU pages, only half the pilots re- 
sponded, and no single item of significance stood out (Q22). When asked which piece of information they 
would remove from the CDU pages (Q23), two pilots responded that surface winds would likely be “out 
of date” by the time they got to the outer marker, and were too dynamic to be useful, and therefore not 
necessary. 

One pilot commented that the actual time interval displayed on the Approach Spacing page seemed to 
lag and sometimes seemed to contradict the commanded speed. To understand why these two pieces of 
information sometimes did not appear to be in agreement requires an in-depth understanding of how the 
ATAAS algorithm computes these parameters. For a pilot who might not be very familiar with the sys- 
tem, this “apparent” inconsistency could lead to believing that these two sources of information were 
conflicting. Since the actual clearance from ATC is to “follow ATAAS speeds,” and the crew is not 
required to actually maintain a given time interval, the spacing interval at any time is not a necessary 
piece of information for the pilots to have. In a data link environment, it would not be necessary for the 
pilots to manually enter data through the CDU. The identification and final approach speed of the aircraft 
to follow, and the spacing interval to maintain would be uplinked, which would lower the workload for 
the nonflying pilot. 

4.6.4. Navigation Display 

The subject pilots were asked how effective the “spacing position indicator” was in communicating 
whether they were fast or slow, relative to the lead aircraft (Q24). The mean value for this question indi- 
cated that the subject pilots judged this display symbology to be slightly effective, but the high standard 
deviation indicated that there was wide variation in opinions among the subject pilots. When asked to 
comment on its effectiveness, the responses were also varied. These comments about the effectiveness of 
the spacing position indicator were positive: it provided excellent situation awareness; the pilot relied on 
it for “proof’ that the system was working; it was a very good, clear symbol; it was a better indicator of 
relative position than closure or separation. Other pilots indicated that they could have flown without it, 
and it was only included in their instrument scan at a low level. 

All subject pilots gave high ratings to the effectiveness of the commanded speed in communicating 
whether the ownship was fast or slow relative to the lead aircraft (Q26), and the “flashing box” used to 
communicate a speed change (Q29), although one pilot commented that it should be bigger to avoid 
confusion with ground speed. Most pilots also gave high ratings for the effectiveness of the green outline 
used to highlight the lead aircraft symbol (Q27). Two additional comments were received, one stating that 
it might pose a problem with a lot of green from weather radar in the background. Another commented 
that it would be better to display the lead aircraft speed under its call sign to make it easier to determine 
the relative speed. 

High ratings were given for the acceptability of the amount of symbology on the display (Q30), indi- 
cating that they did not consider it cluttered. When asked what information they would add to the display, 
three pilots asked for the speed of the lead aircraft, two for the actual time in-trail, and one for wind data. 
When asked what information they would remove from the display, two pilots indicated that they would 
remove the spacing interval target (because it is the target speed only; in a nominal case, it does not 
change unless the spacing clearance changes), and one pilot mentioned the spacing position indicator. 
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4.6.5. Electronic Attitude Director Indicator (EADI) 

The pilots were asked to rate the effectiveness of the F/S Indicator on the EADI in communicating the 
relative speed of the two aircraft. The mean value for this question indicated that the subject pilots judged 
this display symbology to be borderline effective, but the high standard deviation indicated that there was 
much variation among the ratings. One pilot commented that he thought it had some “glitches.” When 
asked which piece of symbology they would remove, two pilots mentioned the F/S Indicator. 

Most pilots rated the effectiveness of the commanded speed (above the F/S Indicator) considerably 
higher (Q36, M = 5.8, SD = 1.0), with only one “borderline” rating, and the rest rated it higher. Two of 
the pilots indicated that they would like to see the commanded speed blink longer, and one of them sug- 
gested it blink until the speed was changed. 

4.6.6. Training 

The subject pilots gave high ratings to the effectiveness of the training received. They indicated that 
they did not think ATAAS training could be accomplished with only paper or video instructions or with- 
out practice in a high-fidelity simulator. When asked specifically what would be the minimum training 
needed to accomplish the task, most pilots indicated that they at least wanted video or computer-based 
(interactive) training. More than half also said they would include fixed-base or procedures simulator 
training. The amount of display symbology (Q37) and the effectiveness of the symbology annunciating 
the current mode of operation (Q38) were both judged by the subject pilots as very effective. 

4.6.7. General Comments 

This section summarizes some of the general written comments from the subject pilots. 

“Overall really liked the concept; but I think more time should be spent during pre-brief on the subject 
of mode switching, capture logic.” (This pilot also had questions about ATC acceptance/notification of 
speeds higher than the profile speed.) 

“...felt that there was a great increase in flight crew situation awareness, and system would provide for 
much more accurate speed and in- trail spacing planning...” (this pilot also noted that he hoped the pro- 
gram would be fully implemented, with RNAV arrivals that have hard airspeed and altitude restrictions). 

One pilot suggested making data input (on CDU) a 2-step procedure (concur-execute), and was con- 
cerned that the other pilot might not catch an error. 

Pilot was somewhat confused with paper-only information until he saw it in action, but then thought it 
was “. . .a great tool and should be used.” 

Pilot commented that the “system would require minimal training from a pilot’s viewpoint...” and 
“thought ATAAS was very user friendly.” 

Pilot thought that “the program was right on target.” Pilot also commented that you need to “integrate 
the ATC clearances without reinventing” the procedures. 

Pilot suggested that the flashing box around the newly commanded airspeed could remain flashing 
until the MCP speed was moved to match, thus ensuring the change didn’t occur while the crew’s atten- 
tion is diverted. He also suggested including flap speed protection. 
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Another pilot thought the concept and software were good, but expressed doubts about how the system 
would operate in the real world with “ 6 or 12 aircraft changing speeds at various rates.” This pilot also 
expressed concern about having speed protection as the flap configuration changes, and suggested that the 
MCP speed window should blank during the Automatic mode. Pilot also mentioned that the workload on 
PNF would increase a great deal if the runway is changed, and the ownship is required to start spacing off 
a new aircraft. 

4.7. Recommendations for Further Improvements 

Implementing the ATAAS tool with a data link interface for receiving clearance data (lead aircraft 
identification (ID) and final approach speed, spacing interval) would help simplify necessary interactions 
with the automation. Modifications could be made to the ND display to simplify training, and would 
eliminate the minor aspects of the tool that some pilots found confusing. 

Addition of wind data to the ADS-B message could improve the accuracy and consistency of the algo- 
rithm’s performance in the presence of changing winds. In an actual flight test, the ATAAS performance 
results could be degraded from those achieved in this study due to the presence of sometimes rapidly 
changing winds. 

Addition of a capability to merge aircraft from different arrival streams into a single stream would ex- 
pand the usefulness of the ATAAS tool because it is very common in today’s environment for aircraft to 
arrive from different directions into a common merge point. 

5.0. Concluding Remarks 

A concept for providing airborne-managed, in-trail spacing in the terminal area was developed and 
evaluated in a full-workload simulator with airline subject pilots. This concept included procedures for 
flight crew interaction with air traffic controllers as well as with the onboard algorithm that provides 
speed commands for achieving a target spacing. Three methods of speed control were evaluated through 
comparison with a baseline case, in which current-day flight procedures were used. 

The main objectives of this study were to evaluate the concept from the standpoint of pilot acceptabil- 
ity, head-down time and workload, and to validate the results seen in a Monte Carlo analysis of the 
Advanced Terminal Area Approach Spacing (ATAAS) algorithm. The subject pilots generally rated the 
workload level with the ATAAS procedure as similar to that with standard procedures. They also rated 
most aspects of the procedure highly in terms of acceptability (though it should be noted that the scenar- 
ios reflected a nominal environment, with no winds or abrupt changes in lead aircraft speed). 

The subject pilots indicated that the head-down time was slightly higher but acceptable when using the 
ATAAS tool. Eyetracker data showed no significant change in the amount of time spent looking out the 
window when ATAAS commands were used. 

Data showed the aircraft was able to consistently achieve the target spacing interval when the ATAAS 
speed command was autothrottle-coupled (Automatic condition). With the speed controlled by the pilot 
through the Mode Control Panel or manual throttle inputs, the mean spacing interval was slightly greater, 
but the consistency (standard deviation) was on the same order as in the Automatic condition. This effect 
is a result of pilot response to the annunciated speed commands and could be mitigated by training or 
display changes to encourage the pilots to follow the ATAAS speed guidance more closely. 
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The subject pilots indicated that the simple pilot and controller procedures developed to accompany 
the tool were highly acceptable. Subject pilots felt that training in a fixed-base simulator is necessary to 
provide them with the knowledge and capabilities needed to perform this type of procedure. 
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Appendix A 

INSTRUCTIONS TO SUBJECT PILOTS, ATAAS, OCTOBER, 2001 


FLIGHT MANUAL BULLETIN 


ADVANCED TERMINAL AREA APPROACH SPACING (ATAAS) 

BACKGROUND INFORMATION 

A method for airborne management of spacing between arriving aircraft in the terminal area has been 
developed and is being tested in simulation. This method employs an airborne algorithm known as the 
Advanced Terminal Area Approach Spacing (ATAAS) algorithm, that compares flight profiles of two 
aircraft and computes a target speed for the following aircraft to maintain. The ATAAS algorithm allows 
the flight crew to accurately maintain an assigned interval from the aircraft sequenced ahead of it, and has 
the potential to increase arrival capacity because of increased spacing accuracy. Use of this algorithm 
requires the flight crew to follow a set of procedures that includes adhering to ATAAS-generated speed 
guidance cues. 

The Air Traffic Service Provider (ATSP) Approach Control will issue a clearance for approach spacing 
after entry into the terminal area. The crew will be responsible for selecting the assigned traffic to follow 
from a displayed list, activating the system, and maintaining the commanded speed guidance. 

A charted arrival procedure that includes a transition to an ILS approach is used to define the lateral path 
for the aircraft to maintain. This arrival procedure is defined such that it can also be used by aircraft that 
are not ATAAS-equipped. Non-equipped aircraft cleared for this arrival must follow the charted speed 
profile and their spacing will be managed by ATSP. 

The ATAAS algorithm is based on a profile similar to the charted speed profile, which emulates the 
speeds Approach Control normally issues to arriving aircraft. This nominal speed profile may be enabled 
by engaging the ATAAS system without first selecting a target. 

PHRASEOLOGY 

Unlike most ATC clearances, the approach spacing clearance requires the use of two call signs, that of the 
following aircraft and that of the lead aircraft. To reduce the possibility for confusion when an approach 
spacing clearance is issued, the recommended phraseology should be strictly followed. 


Clearance for approach spacing: 


ATC: “( Following AC ID) spacing interval is seconds behind (Lead AC call sign) maintain ATAAS 

speeds”. 
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A/C: “Approach, (Following AC ID), spacing interval is seconds behind (Lead AC call sign) maintain 

ATAAS speeds”. 

CREW PROCEDURES 


There are two separate procedure clearances related to the approach spacing operation. The first is for a 
Standard Terminal Arrival Route (STAR) charted procedure that transitions onto the ILS approach and is 
normally issued by the ATSP well before entry into the terminal area. This clearance requires the flight crew 
to follow a lateral and vertical path, and follow a speed profile. The second clearance is for the approach 
spacing operation, as discussed in the phraseology section. The flight crew enters the appropriate inputs on the 
CDU, assesses the commanded speed guidance, advises ATC, and follow the ATAAS speed guidance 
(assuming the clearance is accepted). 

The crew interfaces with the ATAAS algorithm through the Approach Spacing pages on the FMC CDU. 
These pages are accessed by pressing the ATC function key. There are two pages associated with this 
operation. The first of these is used to select whether to operate in spacing mode or profile speed mode 
(see Figure 1). If operating in spacing mode, the lead aircraft identification and time interval to maintain 
is also entered on this page. The second page is used to enter additional data needed to improve the 
performance of the ATAAS algorithm (see Figure 2). Data may be entered on this page at any time. 

Data to be entered on this page are: 

• Own aircraft final approach speed 

• Lead aircraft final approach speed 

• Minimum distance (if different from minimum wake vortex separation distance) 

• Airport surface winds (obtained from ATIS). 


The ATAAS speed guidance is coupled to autothrottles with the ATAAS Speed button located on the left side 
Navigation Display control panel, or can be left in advisory mode and flown with speed controlled either from 
the Mode Control Panel or manually. 



f 

APPR 

SPACING l/i ' 




APPR DATA 




SELECT LEAD 


□ 


NASA557 135 KTS 

□ 

<PROF 

SPEED 

UAL903> 

□ 


UAL903 140 KTS 


□ 



AAL143> 

□ 

□ 



□ 



SWA281> 

□ 

□ 


MIN DISTANCE 



AAL225> 

□ 


4 NM 

□ 



□ 


180 /19 



□ 



COA285> 

□ 

□ 

<APPR 

SPACING 

□ 



APPR DATA> 

□ 

□ 






J 




J 


□ 

□ 

□ 

□ 

□ 

□ 


Figure 1. Approach spacing CDU page Figure 2. Approach Data CDU page 

To operate in profile speed mode, the crew selects the prompt for PROFILE SPEED on the APPROACH 
SPACING page. Additional data can be entered on the APPROACH DATA page. However, since the aircraft 
is not maintaining spacing from a lead aircraft, there is no entry for lead aircraft final approach speed or 
minimum spacing distance. 
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ATAAS MODE ANNUNCIATION 


The ATAAS system is an additional speed mode and will be annunciated on the EADI in the 
conventional manner: 

ATAAS guidance armed “PDA” - mode annunciation in white 
ATAAS guidance active “PDA” - mode annunciation in green 

Note that if the current aircraft speed is greater than the commanded ATAAS speed, the ATAAS 
mode will go immediately to “active”. 

ALERTS 

No EICAS error messages are associated with the ATAAS system 
There are no aural alerts associated with the ATAAS system. 

If an aircraft is actively spacing behind another aircraft and the ATAAS system determines that the 
minimum separation standard will be lost, the crew will be alerted on the ND with an amber “Minimum 
Distance” arc ahead of the aircraft depicting the minimum separation distance and the words “MIN 
DIST”. 
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ATAAS APPROACH 


NOTE 

• ATAAS approaches are not authorized with an engine inoperative or any system irregularity that 
might compromise the operation 

• Maintain speed guidance with autothrottle coupled, with MCP control of autothrottle, or with manual 
control of throttles 

• Expect normal approach clearance 


PNF ATIS information SS ARR/APPR 

C, FO ARRIVAL/APPROACH BRIEF 

PNF SPACING CLEARANCE SET 

• Select traffic to follow 

• Enter spacing interval 


• Enter own and traffic Final Approach Spds 

• Enter minimum distance (if different from standard wake-vortex separation) 

• Enter airport winds 

PF determines if spacing clearance can be accepted (current interval between -30 sec and +90 sec) 

PF, AUTOTHROTTLE (AS DESIRED) ON 

PF ATAAS GUIDANCE SELECTED 

ND shows traffic ID and spacing interval. EADI shows “PDA”, commanded speed 
PNF, SPACING CLRNCE.... ACKNOWLEDGE 
PF, PNF MONITOR 

If amber MIN DIST arc appears, ensure close adherence to ATAAS speed guidance and notify ATC 


This bulletin will be removed when the information is incorporated into the Flight Manual 
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Appendix B1 


Post-Run Questionnaire 

Please respond to the following questions with regard to the approach scenario you just completed. 


1. Rate your physical workload level compared to standard approach procedures. 


Much 

Much Lower The Same Higher 

1 2 3 4 5 6 7 

2. Rate your mental workload level compared to standard approach procedures. 

Much 

Much Lower The Same Higher 

1 2 3 4 5 6 7 

3. Rate your overall workload level compared to standard approach procedures. 

Much 

Much Lower The Same Higher 

1 2 3 4 5 6 7 


4. How acceptable was the amount of heads down time for the following phases of the approach 
procedure? 



Not at all 
Acceptable 



Borderline 



Very 

Acceptable 


1 

2 

3 

4 

5 

6 

7 

Downwind 








Base 








Final 









5. How acceptable was the ATAAS tool during the following phases of the approach procedure: 



Not at all 
Acceptable 



Borderline 



Very 

Acceptable 


1 

2 

3 

4 

5 

6 

7 

Downwind 








Base 








Final 
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6. How confident were you with the guidance provided by the ATAAS tool during the following phases 
of the approach procedure: 



Not at all 
Confident 



Borderline 



Very 

Confident 


i 

2 

3 

4 

5 

6 

7 

Downwind 








Base 








Final 









7. How comfortable were you with using the ATAAS tool during the following phases of the approach 
procedure: 



Not At All 
Comfortable 



Borderline 



Very 

Comfortable 


1 

2 

3 

4 

5 

6 

7 

Downwind 








Base 








Final 









8. Please consider the information sources you used while performing self-spacing related tasks. 
Estimate the relative amount of time you spent obtaining ATAAS spacing guidance information using 
each of the four display features listed below. The total should equal 100%. 


Feature 

% Time 

Spacing position indicator with lead aircraft and 
ownship symbols on ND 


Commanded speed on ND 


Fast/slow indicator with speed bug on EADI 


Commanded speed on EADI 


Total 

100% 
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Appendix B2 


Post-Experiment Questionnaire 


Advanced Terminal Area Approach Spacing (ATAAS) 


January 2002 

Katrin Helbing, Paul Mafera, and Rich Adams 
Booz Allen Hamilton 

For: 

Rosa Oseguera-Lohr, Gary Lohr, and Terence Abbott, 
NASA Langley Research Center 
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Acceptability/F easibility 

1. Overall, how comfortable were you in using the ATAAS tool to maintain the assigned spacing in the 
terminal area? 


Completely 



Borderline 



Completely 

Uncomfortable 






Comfortable 

1 

2 

3 

4 

5 

6 

7 


2. Overall, how effective was the ATAAS tool in helping you maintain the assigned spacing? 


Completely 



Borderline 



Completely 

Ineffective 






Effective 

1 

2 

3 

4 

5 

6 

7 


3. Overall, how does the level of safety provided by the ATAAS tool compare to standard approach 
procedures? 

Compromised Unchanged Enhanced 

1 2 3 4 5 6 7 


4. Overall, how confident were you in the speed guidance provided by the ATAAS tool? 


Not at all 



Borderline 

Confident 




1 

2 

3 

4 


Very 

Confident 

7 


5. How comfortable do you think other pilots would be using a tool similar to ATAAS to maintain 
assigned spacing in the terminal area? 


Completely 



Borderline 



Completely 

Uncomfortable 






Comfortable 

1 

2 

3 

4 

5 

6 

7 


6. How effective do you think a tool such as the ATAAS will be at increasing arrival capacity? 


Completely 



Borderline 



Completely 

Ineffective 






Effective 

1 

2 

3 

4 

5 

6 

7 


7. How well do you think the approach spacing procedures and tools could be integrated into the current 
flight deck operational environment? 


Not At All 



Borderline 



Very 

Effectively 






Effectively 

1 

2 

3 

4 

5 

6 

7 
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8. Can you think of any real world situations where the ATAAS procedures and tools may NOT be 
effective? 


Procedures 


9. Please rate the overall procedure for using ATAAS guidance system. 


Very Difficult 

1 2 


Moderate 

3 4 5 


Very Easy 
6 7 


10. How comfortable were you with the self-spacing clearance procedure? 


Completely 



Borderline 



Completely 

Uncomfortable 






comfortable 

1 

2 

3 

4 

5 

6 

7 


1 1 . How clear was the phraseology used for approach spacing clearances? 


Extremely 



Borderline 



Extremely 

Unclear 






clear 

1 

2 

3 

4 

5 

6 

7 


12a. How comfortable were you in accepting the assigned spacing interval? 


Completely 



Borderline 



Completely 

Uncomfortable 






comfortable 

1 

2 

3 

4 

5 

6 

7 


12b. If you marked 3 or lower on Question 12a, please indicate how the spacing interval posed a problem 
or how it may be improved. 
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13. Compared to standard approach procedures, the amount of time spent looking at displays for the 
approach spacing procedure was: 

Much higher The same Much lower 

1 2 3 4 5 6 7 

14. Can you think of any way of improving the approach spacing clearance procedure? 


Flight Management Computer (FMC) Control Display Unit (CDU) 

Please answer the following questions as they pertain to your simulator experiences with the CDU 
ATAAS pages. 

15. How confident were you in the process of entering data into the CDU ATAAS pages? 


Not At All 



Borderline 


Confident 





1 

2 

3 

4 

5 


Completely 

Confident 


16. How well do the CDU ATAAS pages provide clear (i.e., unambiguous) and consistent labeling? 


Completely 



Borderline 



Completely 

Inadequate 






Adequate 

1 

2 

3 

4 

5 

6 

7 


17. How well do the CDU ATAAS pages allow for recovery from input errors? 


Completely 



Borderline 



Completely 

Inadequate 






Adequate 

1 

2 

3 

4 

5 

6 

7 


18. The method for selecting the lead aircraft was: 


Completely 



Borderline 



Completely 

Inadequate 






Adequate 

1 

2 

3 

4 

5 

6 

7 
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19. The method for entering the spacing interval was: 


Completely 



Borderline 



Completely 

Inadequate 






Adequate 

1 

2 

3 

4 

5 

6 

7 


20. The method for entering additional data (e.g., final approach speed, minimum distance, surface 
winds) was: 


Completely 



Borderline 



Completely 

Inadequate 






Adequate 

1 

2 

3 

4 

5 

6 

7 


21 . How acceptable was the amount of time spent working with the CDU ATAAS pages? 


Completely 



Borderline 



Completely 

Unacceptable 






Acceptable 

1 

2 

3 

4 

5 

6 

7 


22. If you could add one piece of information that is not currently available to the CDU ATAAS pages, 
what would it be? 


23. If you could remove one piece of information from the CDU ATAAS pages, what would it be? 


Navigation Display 

Please answer the following questions as they pertain to your simulator experiences with the Navigation 
Display (ND). Refer to the figure provided. 


24. How effective was the spacing position indicator in communicating whether ownship was 'fast’ or 
'slow' with regard to the Lead aircraft? 


Completely 



Borderline 



Completely 

Ineffective 






Effective 

1 

2 

3 

4 

5 

6 

7 
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25. If the spacing position indicator posed a problem, or if some aspect of the indicator stands out as 
being particularly effective, please comment on how it impacted acceptability, performance, and/or 
workload. 


26. How effective was the commanded speed in communicating whether ownship was 'fast' or 'slow' with 
regard to the Lead aircraft? 


Completely 



Borderline 



Completely 

Ineffective 






Effective 

1 

2 

3 

4 

5 

6 

7 


27. How effective was the highlighting of the lead aircraft (i.e., green outline of the aircraft symbol)? 


Completely 



Borderline 



Completely 

Ineffective 






Effective 

1 

2 

3 

4 

5 

6 

7 


28. If you marked 3 or lower on Question 28, please indicate how the highlighting posed a problem or 
how it may be improved. 


29. The method used to indicate a required speed change (flashing box around commanded speed) was: 


Completely 



Borderline 



Completely 

Ineffective 






Effective 

1 

2 

3 

4 

5 

6 

7 


30. The amount or density of symbology (too much is called ’ clutter') on the ND was: 


Completely 



Borderline 



Completely 

Unacceptable 






Acceptable 

1 

2 

3 

4 

5 

6 

7 
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3 1 . If clutter was a problem on the ND, please comment on what was responsible for the clutter. 


32a. Did you ever see the minimum distance warning? 
No Yes If so: 


32b. How effective was the display in communicating a minimum distance warning? 


Completely 



Borderline 



Completely 

Ineffective 






Effective 

1 

2 

3 

4 

5 

6 

7 


33. If you could add one piece of information that is not currently available on the ND, what would it 
be? 


34. If you could remove one piece of information from the ND, what would it be? 


Electronic Attitude Director Indicator 

Please answer the following questions as they pertain to your simulator experiences with the Electronic 
Attitude Director Indicator (EADI). Refer to the figure provided. 


35. How effective was the fast/slow indicator with speed bug on the EADI in communicating whether 
ownship was 'fast' or 'slow' with regard to the Lead aircraft? 


Completely 



Borderline 



Completely 

Ineffective 






Effective 

1 

2 

3 

4 

5 

6 

7 
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36. How effective was the commanded speed on the EADI in communicating whether ownship was 'fast’ 
or 'slow' with regard to the Lead aircraft? 


Completely 



Borderline 



Completely 

Ineffective 






Effective 

1 

2 

3 

4 

5 

6 

7 


37. The amount or density of symbology (too much is called ' clutter') on the EADI was: 


Completely 



Borderline 



Completely 

Unacceptable 






Acceptable 

1 

2 

3 

4 

5 

6 

7 


38. How effective was the method (e.g., color coding, 'speed bug' shape) for indicating the current mode 
(e.g., armed, engaged) of operation? 


Completely 



Borderline 



Completely 

Ineffective 






Effective 

1 

2 

3 

4 

5 

6 

7 


39. If you could add one piece of information to the EADI that is not currently available, what would it 
be? 


40. If you could remove one piece of information from the EADI, what would it be? 
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Training 


41 . How effective do you feel the training was that you received on the ATAAS tool? 


Completely 



Borderline 



Completely 

Ineffective 






Effective 

1 

2 

3 

4 

5 

6 

7 


42. Do you think ATAAS training could be accomplished with only paper instructions (e.g., user’s 
manual, display diagrams and approach charts, screen shots of sample scenarios, etc.)? 

Definitely Maybe Definitely 

Not 

1 2 3 4 5 6 7 


43. Do you think ATAAS training could be accomplished with only video instructions (e.g., introduction 
to displays and approach charts, video of sample scenarios, etc.)? 

Definitely Maybe Definitely 

Not 

1 2 3 4 5 6 7 


44. Do you think ATAAS training could be accomplished without practice in a high fidelity aircraft 
simulator? 

Definitely Maybe Definitely 

Not 

1 2 3 4 5 6 7 


45. If a system similar to the one you just used were to be integrated into your aircraft, what is the 
minimum training you would need to accomplish the task (think in terms of paper, video and simulator 
instruction)? 
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Comments 


Please use the space below to address any additional comments related to the ATAAS task or study you 
just completed. 
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Appendix C 


NASA TLX Scale 

Mental Demand- How much mental and perceptual activity was required (e.g., thinking, deciding, 
calculating, remembering, looking, searching, etc.)? 

Physical Demand - How much physical activity was required (e.g., pushing, pulling, turning, controlling, 
activating, etc.)? 

Temporal Demand - How much time pressure did you feel due to the rate or pace at which the tasks or 
task elements occurred? 

Performance - How successful do you think you were in accomplishing the goals of the task set by the 
experimenter (or yourself)? How satisfied were you with your performance in accomplishing these goals? 

Effort - How hard did you have to work (mentally and physically) to accomplish your level of 
performance? 

Frustration - How insecure, discouraged, irritated, stressed and annoyed versus secure, gratified, content, 
relaxed and complacent did you feel during the task? 

MENTAL DEMAND 

i I i I i I i I i I i I i I i I i I \ m 

Low High 

PHYSICAL DEMAND 

i I i I i I i I i I i I i I i I i I i 

Ltnv High 

TEMPORAL DEMAND 

i I i I i I i I i I i I i I i I i I i 

Ltnv High 

EFFORT 

i I i I i I i I i I i I i I i I i I i 

Ltnv High 

PERFORMANCE 

i I i I i I i I i I i I i I i I i I i 

Good Poor 

FRUSTRATION 

i I i I i I i I i I i I i I i I i I i 

Loiv High 
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